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(54) Graphical user interface for devices having small tactile displays 



(57) Apparatus, such as personal digital assistants, 
mobile phones or mobile computers, is described hav- 
ing a touch sensitive display and circuitry responsive to 
the display to move a cursor according to movement of 
a finger thereon and effect input operations according 
to the position of a cursor in relation to a displayed im- 
age, the position of the cursor on the displayed image 



being displaced by a short distance from the point of 
contact of the finger with the display so that the position 
of the cursor when an input operation is effected is vis- 
ible to the user. The input operations comprise at least 
a first finger tap serving to define the position of the cur- 
sor and a second finger tap serving to confirm the posi- 
tion of the cursor as the point of effect desired by the 
user 
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Description 
Field of the Invention 

[0001] The present invention relates to user interfac- 
es for electronic devices, particularly but not exclusively, 
personal digital assistants, mobile phones or mobile 
computers, that have small display screens and that em- 
ploy touch sensing as a means of data input. 



Background of the invention 

[0002] Until relatively recently, electronic documents 
and graphical user interfaces have been primarily 
viewed and manipulated in electronic devices on desk- 
top or laptop consoles with relatively large displays, typ- 
ically 15" or 1 7" CRT or flat panel displays or larger and 
data input has been effected using keyboard and mouse 
devices. 

[0003] Due to increasing focus on compactness of 
electronic devices, however, the displays, especially in 
portable electronic devices, are becoming smaller and 
smaller. Popular electronic devices with a smaller dis- 
play area include electronic organizers, PDA's (person- 
al digital assistants), and graphical display-based tele- 
phones. Also available today are communicators that fa- 
cilitate various types of communication such as voice, 
faxes, SMS (Short Messaging Services) messages, e- 
mail, and Internet-related applications. These products 
can likewise only contain a small display area. Often 
such devices do not use a full keyboard or a mouse, 
rather the display screens of these devices are touch 
sensitive to allow data input. A wide variety of gesture- 
based and other user interface techniques have been 
used and proposed to facilitate data entry through the 
touch screen. 

[0004] For instance, US5745116 describes a user in- 
terface for a mobile telephone in which a user performs 
a manual selection or a gesture selection of a screen 
object on a screen using a pointing device. After a man- 
ual selection, such as a single tap, the electronic device 
automatically presents a temporary directional palette 
having palette buttons that explicitly state functions of 
the electronic device. Each palette button has a unique 
compass direction relative to the original tap area. By 
making a second tap on a desired palette button, a nov- 
ice user learns available functions of the electronic de- 
vice and their corresponding directional gestures. Alter- 
nately the user may perform a gesture selection of both 
a screen object and a function, such as making a double 
tap or drawing a line in the appropriate direction, before 
the directional palette appears on the screen. 
[0005] This and many other known touch sensitive 
display screens are accompanied by a stylus to enable 
a more precise location of an input operation on a graph- 
ical user interface than would be possible using a finger, 
which is generally relatively large in comparison with the 
display device and the images displayed on it. 



[0006] However, the use of a stylus has major disad- 
vantages. First the stylus is necessarily a removable 
component for which a way must be provided of fixing 
it to the device. In use it is necessary to remove the sty- 
5 lus from its fixed position, hold it like a pen and then 
replace it in its position after use. It is necessary to take 
care not to lose it - in fact with many products a set of 
replacement styli are provided by the manufacturer. The 
diameter of the stylus is often very reduced, which adds 
10 to the risk of dropping and losing it. Moreover, many us- 
ers are tempted to use a ballpoint pen or other pointed 
implement instead of the stylus, which can wear or dam- 
age the surface of the touch screen. 
[0007] In addition, at least some types of touch pad 
'5 are based on a grid of resistive elements, the spacing 
between which is comparable to the size of the stylus 
and which is usually greater that the pixel resolution of 
the display. This can lead to unreliability in use since in 
practice that point of contact between the stylus and the 
20 screen can fall in the interstices of the resistive matrix. 
[0008] Some products combine a finger input for 
some operations with the use of a stylus for others. This 
adds to the difficulty of using the device since the user 
must continually interchange stylus and finger. 
25 [0009] This invention is intended to mitigate the draw- 
backs of the prior art by providing an interface for such 
devices that does not require a stylus, but rather allows 
input to be effected through small active screen ele- 
ments using a finger alone. 

30 

Summary of the invention 

[0010] According to the present invention, there is 
provided apparatus having a touch sensitive display and 

35 circuitry responsive to the display to move a cursor ac- 
cording to movement of a finger thereon and effect input 
operations according to the position of a cursor in rela- 
tion to a displayed image, the position of the cursor on 
the displayed image being displaced by a short distance 

40 from the point of contact of the finger with the display so 
that the position of the cursor when an input operation 
is effected is visible to the user, wherein the input oper- 
ations comprise at least a first finger tap serving to de- 
fine the position of the cursor and a second finger tap 

45 serving to confirm the position of the cursor as the point 
of effect desired by the user. 

[0011] Large tactile display systems in which a cursor 
is displayed displaced from the point of contact with a 
finger are known, for instance US5808605 describes a 

50 computer system in which a virtual pointing device is 
created by detecting a entire hand placed on the touch- 
screen. Input commands are effected by moving parts 
of the hand. US 4812833 describes a touch panel input 
device that includes a sensor for detecting that an op- 

55 erators finger has approached towards a touch input key 
and displaying a cursor for indicating this key. 
[0012] The present invention makes use of a similar 
technique for enabling a cursor to remain visible and 
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combines it with a double tap mechanism to provide a 
convenient and user-friendly way for small active ele- 
ments - smaller than a human finger - to be actuated by 
a finger. 

[0013] One advantage of this arrangement in at least 
some embodiments is that it enables the resolution of 
the touchpad to be decoupled from the size of the active 
elements, enabling either the size of the latter to be re- 
duced or more satisfactory operation with a coarser res- 
olution of the touchpad. 

[0014] The touchpad needs only to have sufficient 
resolution to enable an effective point of contact with the 
fingerto be determined, whilst the point at which an input 
operation has effect is limited only by the resolution of 
the display and the point at which active elements be- 
come too small to be comfortably visible. The point of 
contact of the finger and the touch pad can be calculated 
for instance from a set of matrix points covered by the 
fingerto a greater resolution than that of the matrix itself. 

Brief Description of the Drawings 

[0015] A personal digital assistant embodying the in- 
vention will now be described, by way of non-limiting ex- 
ample, with reference to the accompanying diagram- 
matic drawings, in which: 

Figure 1 is a schematic diagram showing an elec- 
tronic device having a tactile display; 
Figure 2 shows a personal digital assistant having 
a graphical user interface; 

Figure 3 illustrates the cursor geometry in the user 
interface of Fig 2; 

Figure 4 is a flow diagram showing the operation of 
the user interface. 

Best Mode of Carrying Out the Invention 

[0016] Fig 1 shows in schematic form an electronic 
device according to an embodiment of the invention. A 
touchpad input device 100 for instance of the resistive 
type sold under the TouchTek4 brand by MicroTouch 
Systems, Inc. provides input via a suitable controller (not 
shown) to a computing device 110 that requires input. 
Computer 110 is connected to a display device 120, of 
any suitable type such as an LCD display. 
[0017] As is well known, touchpad input devices are 
small, touch-sensitive devices that can be used as a 
pointing device to replace a mouse, trackball or other 
cursor locator/input device in mouse-driven or other per- 
sonal computers. The touchpad typically includes a 
small touch-sensitive screen up to 3" by 5" in size and 
produces X, Y location coordinates representative of the 
location of the touching device (finger or inanimate ob- 
ject such as stylus) on its surface. The computer 110 
interprets these X,Y coordinates to locate the cursor on 
the computer display. The user controls the computer 
cursor location by moving their finger across the sensor 



surface. Touch pad 100 is transparent and physically 
overlies the display device 120. 

[0018] One example of a device with this general 
structure is the personal digital assistant 200 shown in 

5 Fig 2. PDA 200 comprises a touch sensitive display 220, 
incorporating touch pad 1 00 and display 1 20. A user in- 
terface is displayed on display 220 in order to allow a 
user to effect input operations according to the position 
of a cursor 240 in relation to a displayed image having 

10 active elements that are in general smaller than the fin- 
ger, such as the images of the keys of a keyboard illus- 
trated at 250. Such active elements may of course also 
include icons, scroll bars, dates on a calendar, charac- 
ters in a document or the like. 

15 [0019] As can be seen in Fig 2, the position of the cur- 
sor on the displayed image is displaced by a short dis- 
tance - around 5mm for instance in preferred embodi- 
ments - from the point of contact of the finger 230 with 
the display so that the position of the cursor when an 

20 input operation is effected is visible to the user. This dis- 
placement can be set by the user according to their pref- 
erence and the size of their finger. 
[0020] Fig 3 is a schematic diagram that shows the 
geometrical relationship between cursor 240 and zones 

25 on contact 270 between and finger and touch screen at 
an initial location DO and locations of first and second 
finger taps D1 and D2 respectively. 
[0021] It will be appreciated that with this displace- 
ment between the point of contact that the position of 

30 cursor 240, there is a zone - denoted 260 in Fig 2 - at 
the bottom of the touchpad into which the cursor cannot 
be moved. This zone can either be used to display non- 
active elements, such as a date and time, or the device 
can be arranged so that the touchpad is slightly larger 

35 in this dimension than the underlying display surface. A 
similar zone 261 exists at the top of the screen where 
detection of the point of contact of a finger is unneces- 
sary. 

[0022] Fig 4 shows in section an embodiment in which 
40 touchpad 100 and display screen 120 are laterally dis- 
placed one from another to create zones 260 and 261 . 
[0023] Fig 5 is a flow diagram showing an operating 
process operated by the graphical user interface soft- 
ware that controls display 220 in this embodiment. In 
45 applications to PDA this would be incorporated in the 
operating system of the device. 

[0024] The process starts at step 300 when a finger 
230 touches the screen. Detection of the finger in con- 
tact with the screen for greater than a threshold time - 

so for instance 0.3s - causes cursor 240 to be displayed on 
the screen. This time threshold is designed to filter ac- 
cidental touches. A user can then cause the cursor to 
move on the screen by moving their finger. 
[0025] Once the user has positioned cursor 240 in a 

55 desired location - denoted S in Fig 3 and 4, overlying for 
instance a chosen key in keyboard 250, a chosen icon 
or other active display element, the usertaps the display 
twice in relatively quick succession at that location. The 
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first finger tap 3 1 0 serves to fix the position of the cursor 
and the second finger tap 320 serves to confirm the po- 
sition of the cursor as the point of effect desired by the 
user. 

[0026] First a check is carried out to check whether 5 
the first tap is spatially associated with the cursor posi- 
tion - decision step 330. If DO - D1 is less than a thresh- 
old distance Dm ! where DO is the position of the last con- 
tact point that determined the cursor location and D1 is 
the point of contact of the first tap, then the position of 10 
the cursor is defined as the position S of the last contact 
point. The distance Dm is preferably settable for optimal 
performance, but would typically be set at around 3mm. 
If D0-D1 is greater than the threshold Dm then the cursor 
is simply moved to the position S1 of the first tap - step 15 
350. 

[0027] Next the temporal association of the two taps 
is determined in step 340 - if the time elapsed t2-t1 be- 
tween the two taps is less than a settable threshold Tm 
then the input operation associated with whatever active 20 
element is located under the cursor is carried out. The 
time Tm can be set by the user for optimal performance, 
but could be set to around 0.2s, for instance. 
[0028] If t2 - 11 is greater than tm a check is carried 
out to see if the taps are in the same location - decision 25 
step 350 and 360. If t2 - 11 is greater than the threshold 
tm then the cursor is moved to the position S2 of the 
second tap - step 360. 

[0029] The position of the second tap is not taken into 
consideration in order to allow two different fingers to be 30 
used for the two taps. If the user chooses to actuate the 
device in this way, then the taps would be necessarily 
spatially separated. 

[0030] The above represents relatively simple em- 
bodiment of the invention. It will be understood that 35 
many variations are possible. For instance, extra taps 
may be incorporated afterthe firsttap in order to emulate 
for instance an mouse double click. Various types of vis- 
ual feedback may be given to the user, for instance the 
cursor may change colour, brightness or form once its 40 
position has been fixed by the first tap. A longer time 
threshold may be introduced after which a second tap 
is ineffective regardless of where it takes place. 
[0031] Whilst the invention is particularly useful in 
portable, handheld devices, it will be understood that 45 
technique may be applied to any kind of device, whether 
portable or not, that includes a tactile display, for in- 
stance printers, photocopiers, fax machines as well as 
industrial machinery. 

[0032] Although a specific embodiment of the inven- so 
tion has been described, the invention is not to be limited 
to the specific arrangement so described. The invention 
is limited only by the claims. The claims themselves are 
intended to indicate the periphery of the claimed inven- 
tion and are intended to be interpreted as broadly as the 55 
language itself allows, rather than being interpreted as 
claiming only the exemplary embodiment disclosed by 
the specification. 



Claims 

1 . Apparatus having a touch sensitive display and cir- 
cuitry responsive to the display to move a cursor 
according to movement of a finger thereon and ef- 
fect input operations according to the position of a 
cursor in relation to a displayed image, the position 
of the cursor on the displayed image being dis- 
placed by a short distance from the point of contact 
of the finger with the display so that the position of 
the cursor when an input operation is effected is vis- 
ible to the user, wherein the input operations com- 
prise at least a first finger tap serving to define the 
position of the cursor and a second finger tap serv- 
ing to confirm the position of the cursor as the point 
of effect desired by the user. 

2. Apparatus as claimed in claim 1 wherein the input 
operations are associated with display elements 
that are smaller than a human finger. 

3. Apparatus as claimed in claim 1 or claim 2 wherein 
the input operation is effected if the second finger 
tap is effected less than a predetermined time after 
the first finger tap. 

4. Apparatus as claimed in claim 3 wherein the input 
operation is effected if the first finger tap is effected 
less than a predetermined distance from the contact 
point that determined the cursor position. 

5. Apparatus as claimed in any preceding claim 
wherein the display is arranged not to display active 
elements in an edge zone. 

6. A touch sensitive display for use in apparatus as 
claimed in claim 5 in which in which the touch pad 
extends beyond the edge of the display surface to 
create the edge zone. 

7. A touch sensitive display for use in apparatus as 
claimed in claim 5 or claim 6 in which the display 
surface extends beyond the edge of the touch pad 
at an edge opposite said edge zone. 

8. Apparatus as claimed in claim 5 wherein user inter- 
face software is programmed not to display active 
elements in the edge zone. 

9. Apparatus as claimed in any preceding claim 
wherein the circuitry includes an operating system. 

10. A method for operating a touch sensitive display in- 
cluding circuitry responsive to the display to move 
a cursor according to movement of a finger thereon 
and effect input operations according to the position 
of a cursor in relation to a displayed image, the 
method comprising: 
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displaying a cursor at a position displaced by a 
short distance from the point of contact of the 
finger with the display so that the position of the 
cursor when an input operation is effected is 
visible to the user; responding to a first finger 
tap by defining the position of the cursor; and 
responding to a second finger tap to confirm the 
position of the cursor as the point of effect de- 
sired by the user. 



10 



11. A computer program for controlling apparatus hav- 
ing a touch sensitive display, the program compris- 
ing program elements for responding to signals 
from the display to move a cursor in a displayed im- 
age according to movement of a finger thereon and * 5 
program elements for effecting input operations ac- 
cording to the position of a cursor in relation to the 
displayed image, the position of the cursor on the 
displayed image being displaced by a short dis- 
tance from the point of contact of the finger with the 20 
display so that the position of the cursor when an 
input operation is effected is visible to the user, 
wherein the input operations comprise at least a first 
f ingertap serving to define the position of the cursor 
and a second finger tap serving to confirm the po- 25 
sition of the cursor as the point of effect desired by 
the user. 
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